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(54) Apparatus and method for maintaining integrated data consistency across multiple 
databases 



(57) A database network having a primary database 
and a plurality of heterogeneous subscribing databases 
for replicating data updates of the primary database in 
the heterogeneous subscribing databases includes a 
database engine connected to the primary database for 
capturing the data updates in the primary database, a 



query manager connected to the database engine for 
generating queries,, where each of the queries is trans- 
lated based on a specified format for each of the heter- 
ogeneous subscribing databases, and a data distributor 
connected to the query manager for distributing the 
translated queries to the heterogeneous subscribing da- 
tabases 
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D scription 

Background of the Invention 

This invention relates to synchronization of various 
databases in a network. More particularly, the invention 
relates to synchronization of replicated data across het- 
erogeneous databases in a communication network. 

Many types of databases are known. There are re- 
lational model database, hierarchical model database, 
network model database, and many other database 
models. Among the available database models, the re- 
lational model is the most popular model. 

In the relational database model, the database is 
divided into two areas - physical model and logical mod- 
el where the physical model is based on each entity and 
the logical model is based on entity relationship. The 
physical and logical models are then used to generate 
a physical structure of a relational model database. 

The hierarchical model has data structured in a par- 
ent-child relationship where the data of the child can be 
acc ssed only through the parent. Thus, in hierarchical 
database, the access to a child entity dependent on a 
parent entity can be only accessed if the key leading to 
the parent entity is available. In contrast, in the relational 
model, the data of any entity can be accessed as long 
as there is an appropriate key leading to such database. 

For example, one way to access the relational da- 
tabase is a well-known definer query language called 
Structural Query Language ('SQL' or Sequel). But for 
the hierarchical database, since there is no relationship 
to connect the entities, it requires a very specific function 
call to access each entity by giving the segment key or 
by giving some other type of identifier to access the par- 
ent of the entity. Once the parent is accessed, the child 
segment can be searched or a segment number can be 
directly called on for locating the entity. If the entity is 
still in another child segment the searching continues 
down to the deeper level until the entity is located. 

The network model is similar to the relational model. 
The entities are ail related where the relationship be- 
tween the segments is called owner and membership. 
The owner owns many members and each member 
could be the owner of other members. 

Other database models include proprietary data- 
base model and object-oriented database model. The 
proprietary database model can be of any construction 
that is suited to the use of a particular software vendor. 
Thus, the proprietary model may have any of combined 
features derived relational, hierarchical and network 
models or may have a totally different structure from any 
of the traditional database models. 

Since significant amount of replicated information 
is r sident among the network elements, operation sys- 
tems, billing and support systems; it is important that the 
replicated copi s of primary data are synchronized with 
the primary data. However, keeping consistent replicat- 
ed information becomes more difficult wh re network el- 



em nts can b any type of the database models dis- 
cussed above such as the r lational model, hierarchical 
model or network model. 

In the telecommunication industry, a standard inter- 
s face called Open Database Connectivity ("ODBC") is 
used for communication among different types of plat- 
forms such as IBM mainframes and Unix workstations. 
However, ODBC is not useful if the databases of these 
different platforms are different type. For example, when 
the primary database is relational and the subscribing 
databases are hierarchical, the updated data in the pri- 
mary database may not be propagated because the pri- 
mary database uses Structural Query Language 
("SQL") while the subscribing databases do not recog- 
nize SQL. 

The data inconsistency among these database 
models, if occurs, not only causes a significant revenue 
loss from a business standpoint, but also requires ex- 
tremely high cost manual data synchronization. Further, 
the manual data synchronization involving human serv- 
ice may cause more data inconsistency problems. 

Currently available systems for updating subscrib- 
ing databases are designed for homogenous databas- 
es. The current systems also use the protocol called 
"two-face commit. ■ The two-face commit means that the 
updating of subscribing databases occur only when all 
of the subscribing databases can be updated. Although 
this type of system provides that all subscribing data- 
bases at all times contain same information, the data- 
base updates may not be propagated to the databases 
that need the updates due to the failed acknowledgment 
from the databases that do not need the updates. Also, 
if a single subscribing database fails to acknowledge an 
update availability, it prevents all the rest of the subscrib- 
ing databases from receiving the database updates. 

It is therefore an object of this invention to provide 
an efficient system and method that accurately and 
promptly synchronize heterogeneous databases in a 
network. 

It is also an object of this invention to provide a sys- 
tem and method that correctly propagates the updates 
in the primary data only to subscribing databases that 
need the updates. 

Summary of the Invention 

These and other objects of the invention are accom- 
plished in accordance with the principles of the invention 
by providing a database network having a primary da- 
50 tabase and a plurality of heterogeneous subscribing da- 
tabases for replicating data updates of the primary da- 
tabase in the heterogeneous subscribing databases. 
The database network includes a primary database en- 
gine connected to the primary database for capturing 
55 the data updates in the primary database. A query man- 
ager connected to th database engine for gen rating 
queries translates queries based on a specified format 
for each of the heterogen ous subscribing databases. 
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A data distributor connected to the qu ry manager dis- 
tributes the translat d queries to the het rogen ous 
subscribing databases. 

The database network may further include a sub- 
scription controller for verifying that at least one of said s 
subscribing databases needs the updates made in the 
primary database so that the query manager generates 
queries only for the subscribing databases that need the 
updates. 

The database network may also include a synchro- *o 
nization manager for controlling sequences of the queue 
distribution where each queue is categorized as imme- 
diate or deferred queue based on query's time sensitiv- 
ity. The queue may be categorized as a retry queue if 
the previous transmission has failed. is 

Further features of the invention, its nature and var- 
ious advantages will be more apparent from the accom- 
panying drawings and the following detailed description 
of the preferred embodiments. 

20 

Brief Description of the Drawings 

FIG. 1 is a simplified block diagram of illustrative 
network which can be operated in accordance with this 
invention. 2s 

FIG. 2 is a more detailed block diagram of illustra- 
tive integrated data consistency server of the network 
of FIG. 1. 

FIGS. 3-5 are a flow chart of steps for carrying out 
an illustrative embodiment of the method of this inven- 30 
tion. 

Detailed Description of the Preferred Embodiments 

In the illustrative embodiment shown in FIG. 1, da- 35 . 
tabase network 100 of the present invention includes 
primary system 102 connected to subscribing systems 
104 via communication network 106. 

Primary system 102 contains primary application 
108, primary database engine 110, primary database 40 
112and integrated data consistency server 114. Primary 
database engine 110 detects the transactions in primary 
application 108 that causes updates in primary data- 
base 112 and captures the updates to forward to inte- 
grated consistency server 114. Upon the receipt of the <s 
updates, integrated consistency server 114 runs the 
steps illustrated in FIGs. 3 and 4, while the update da- 
tabase transactions are performed at primary database 
112. 

Communication network 106 delivers the proc- so 
ess d results from integrated data consistency server 
114 to proper database engines 116, 118 and 120 of 
subscribing systems 104. For example, subscribing sys- 
tems 104 include database engines 116, 118 and 120 
attached to hierarchical databas 122 : r lational data- ss 
bas 1 24 and network database 1 26. Many other typ s 
of databas s can be added to subscribing systems 1 04. 
Database engine 116. 118 and 120 each recerv s ap- 
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propriately formatt d databas updates and updates hi- 
erarchical database 122, relational database 124 and 
network database 126 respectively. 

FIG. 2 shows integrated data consistency server 
114 (FIG. 1 ) connected to primary database engine 110 
(FIG. 1) and communication network 106 (FIG. 1) in 
more details. Integrated data consistency server 114 in- 
cludes subscription controller ("SC") 200, query manag- 
er COM') 204, update queues ("Un") 206, queue con- 
trollers ("Qc") 208, concurrence controller ("CC") 210, 
data distributor ("DD") 21 2 and response handler ("RH") 
214. 

Subscription controller 200 is connected to data- 
base engine 110 and receives the data updates cap- 
tured or recorded by primary database engine 110. 
Based on the captured data updates, subscription con- 
troller 200 determines whether the data updates need 
to be propagated at all. The data updates need to be 
propagated if any of databases 1 22, 1 24 and 1 26 needs 
the data updates for its application. 

Based on the findings of subscription controller 200, 
query manager 204 connected to subscription controller 
200 generates queries only for the databases that need 
the updates. Query manager 200 then determines 
which of subscribing databases requires translations 
and subsequently translates the queries into appropri- 
ate formats based on the types of subscribing database 
models. Query manager 202 is connected to synchro- 
nization manager 204 so that when a query is generat- 
ed, a notice alerts synchronization manager 204. 

Update queues 206 connected to both query man- 
ager 204 and synchronization manager 204 store the 
queries generated from query manager 204. Each 
queue also contains the number of update transactions 
(■Un") which indicate the total update transactions 
stored in each update queue. 

Each of queue controller 208 is connected to each 
of corresponding update queues 206 and synchroniza- 
tion manager 204. Thus, the number of queuing control- 
lers indicate the number of present queues waiting for 
transmission to subscribing system 104. 

Concurrence controller 210 connected to queue 
controller 208 and synchronization manager 204 con- 
trols the openings of the distribution channel of data dis- 
tributor 212 and contains the control level ( m C\ m ) infor- 
mation which indicate the number of available distribu- 
tion channels. 

Data distributor 21 2 is connected to both concurrent 
controller 21 0 and synchronization manager 204 distrib- 
utes updates in queues to subscribing systems 104. 

Communication network 106 connects data distrib- 
utor 212 and subscribing systems 104. Communication 
network 106 may contain various types of communica- 
tion means such as underground telecommunication 
links, wireless links and satellite links. Communication 
network 1 06 may also contain subnetwork such as com- 
puter network or cable television network. 

The functions of subscription controller 200, qu ry 
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manager 204, update queu 206, queu controller 208, 
concurrence controller 210, data distributor 212 and re- 
sponse handler 214 are also illustrated with respect to 
steps in FIGs. 3 and 4. 

FIG. 3 shows an illustrative sequence of steps in 
accordance with this invention tor updating the subscrib- 
ing systems 104 (FIG. 1) with the data updates made in 
primary system 102 (FIG. 1). However, the order ot the 
steps is not critical and can be arranged in various other 
ways without compromising the operation of the heter- 
ogeneous database network 100 (FIG. 1) of present in- 
vention. 

In step 300, primary application 108 (FIG. 1) proc- 
esses transactions that require database updates in pri- 
mary database 112 (FIG. 1). 

In step 302, primary database engine 110 (FIG. 1) 
detects the database update transactions and captures 
the data updates to be made in primary database 112 
(FIG. 1). 

In slep 304, primary database engine 110 passes 
the captured data updates to subscriber controller 200 
(FIG. 2) while primary database 1 1 2 (FIG. 1 ) is updated. 

In step 306, subscriber controller 200 (FIG. 2) de- 
termines subscription of subscribing systems 104 (FIG. 
1), i.e. whether any database in subscription systems 
104 (FIG. 1) is affected by the update transactions. Sub- 
scriber controller 200 (FIG. 2) thus determines if any of 
subscribing databases 122, 124 and 1 26 needs the up- 
dates that are made in primary database 112 (FIG. 1). 

If none of subscribing databases 122, 124 and 126 
(FIG. 1 ) needs the updates, integrated data consistency 
server 1 14 (FIG. 1 ) becomes idle and returns to step 302 
to wait for the detection of database engine 110 (FIG. 
1 ) for next update transaction in primary system 102 
(FIG. 1) 

If any subscribing databases 122, 124 and 126 
(FIG. 1) needs the updates of primary database 112 
(FIG. 1). then query manager 202 (FIG. 2) in step 308 
receives the updates. 

In step 310, query manager 202 (FIG. 2) retrieves 
registration data from primary database 112 (FIG. 1) or, 
alternatively, a separate database (not shown) contain- 
ing registration data. The registration data contains in- 
formation about databases 122, 124 and 126 (FIG. 1) 
of subscription system 104 (FIG. 1 ). The registration da- 
ta also contains predetermined or sample query de- 
scriptions that can be used by query manager to con- 
struct queries appropriate for each of subscribing data- 
bases 122, 124 and 126 (FIG. 1). 

In step 312, query manager 202 (FIG. 2) generates 
a database query for each of databases 122, 124 and 
126 (FIG. 2) in subscription system 104 (FIG. 1) based 
on the retrieved registration data. The database queries, 
for xarnple, is constructed in the standard Structured 
Qu ry Language ("SQL") designed for r lational data- 
bas . 

Using the subscriber database information, query 
manager 202 (FIG. 2) performs query translations of a 



g neral query receiv d from primary database engine 
110 (FIG. 1) to a query that is recognizable by subscrib- 
ing databases 122, 124 and 126 (FIG. 1). The format for 
translations is defined by subscriber engines 116 : 118 
5 and 120 (FIG. 1). 

Synchronization manager 204 (FIG. 2) in step 314 
determines whether the query has an immediate mode 
i.e. requiring immediate attention so that the query is 
distributed to subscribing systems 104 (FIG. 1 ) prompt- 
to ly. 

If not, the synchronization manager 204 (FIG. 2) 
sets up a date and time interval for each query so that 
queue containing such query can be distributed at the 
prespecrfied date and time. 

1$ In step 318, synchronization manager 204 (FIG. 2) 
in step 318 stores the translated queries in update 
queues 206 (FIG. 2) where the query with the immediate 
mode is placed into an immediate queue and the query 
with specified date and time is placed into a deferred 

20 queue. 

Update queues 206 (FIG. 2) thus fall into any of 
three following categories: immediate queue ("I"), de- 
ferred queue ("D") and retry queue ("R"). The retry 
queue is set when the acknowledgment from subscrib- 
es ing systems (FIG. 1 ) indicates failure in the transmission 
process and described in more details with respect to 
FIG. 5 

In step 320, synchronization manager 204 (FIG. 2) 
confirms that query- has been generated into queues. 
30 If not, the synchronization manager 204 (FIG. 2) re- 
turns to step 312 to place the query into a queue. 

If confirmed, synchronization manager 204 (FIG. 2) 
determines whether there is any more subscribing sys- 
tem 1 04 that needs the query containing the updates in 
35 primary system 1 1 0 ( Fl G . 1 ) 

If there is a subscribing system 104 (FIG. 1) that 
needs the update query, then synchronization manager 
204 (FIG. 2) returns to step 310 to generate query for 
the additional subscribing system(FIG. 1 ) based on the 
40 registration data specific to the additional subscribing 
system 104 (FIG. 1). 

If there no subscribing system 1 04 (FIG. 1 ) requiring 
the updates, synchronization manager (FIG. 2) in step 
324 increases the number of update transactions ("Un") 
45 by one. 

In step 326, synchronization manager 104 (FIG. 2) 
determines that all of the subscribing systems 104 (FIG. 
1 ) that need updates already have translated queries in 
queues waiting to be transmitted to the subscribing sys- 

50 terns 104 (FIG. 1). 

If not, the synchronization manager 104 (FIG. 2) re- 
turns to step 308 so that query manager 202 (FIG. 2) 
review the updates that were received from primary sys- 
tem 102 (FIG. 2). 

55 if all qu ues nec ssary for updating the subscribing 
systems 104 (FIG. 1) are g nerated, synchronization 
manager 1 04 (FIG. 2) in step 328 is triggered to proc ed 
with distribution as described in greater detail with re- 
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spect to FIG. 4. 

Referring to FIG. 4, synchronization manager 204 
(FIG. 2), in step 400, constantly checks the status of the 
queries in queues 206 (FIG. 2) that need to be transmit- 
ted. Synchronization manager 204 (FIG. 2) checks the 
queue status by first determining in step 401 whether 
there is any queue in queue controller 208 (FIG. 2) hav- 
ing the immediate mode. 

If there is any queue having the immediate mode, 
synchronization manager 204 (FIG. 2) in step 402 se- 
lects all the queues with the immediate mode and 
groups them for transmission to subscribing systems 
104 (FIG. 1). 

For each of databases 112, 124 and 126 (FIG. 1), 
synchronization manager 204 (FIG. 2) in step 404 proc- 
esses concurrence check by first determining in step 
406 whether the number of update transactions, Un, is 
greater than or equal to one, i.e., whether there is at 
least one updates in the queue. 

If there is at least one update transaction in the 
queue, synchronization manager 204 (FIG. 2) in step 
408 determines if the number of queries subtracted by 
the concurrence level is greater than 0. Synchronization 
manager 204 (FIG. 2) thus checks whether concurrence 
controller 21 0 (FIG. 2) can handle the number of queries 
handl d by query manager 202. 

If it is found that concurrence controller 210 (FIG. 
2) can handle the number of queries handled by query 
manager 202, synchronization manager 204 (FIG. 2) in 
step 41 0 enables concurrence controller 21 0 (FIG. 2) to 
open the distribution channels for queues and data dis- 
tributor 212 (FIG. 2) distributes update transaction for- 
matted for each of database engines 116, 118 and 120 
(FIG. 1) of subscribing system 104 (FIG. 1) via commu- 
nication network 106 (FIG. 1). 

Synchronization manager 204 (FIG. 2) returns to 
step 400 to determine whether all the queues with the 
immediate mode have been processed. 

If there is no more immediate queue, synchroniza- 
tion manager 204 (FIG. 2) in step 412 selects and 
groups all the retry queues in the update queues 206 
(FIG. 2) 

Synchronization manager 204 (FIG. 2) in step 414 
determines if any retry queue is found from step 412. If 
not, synchronization manager 204 (FIG. 2) proceeds to 
step 422 to process the rest of the queues which would 
be deferred queues at this point. 

If any retry queue is found, synchronization manag- 
r 204 (FIG. 2) in step 416 examines the retry queue by 
verifying that the retry count does not exceed the pre- 
det rmined maximum count number. 

If the retry count exceeds the predetermined maxi- 
mum number of requeries, synchronization manager 
204 (FIG. 2) prompts in step 418 a notice to a system 
administrator r porting that th r try count exceeds th 
maximum number. This st p ensures that the transmis- 
sion r sourc s and queuing r sourc s are not wasted 
on queues that are direct d to missing or defective sub- 



scribing systems. 

If the retry count does not exceed the maximum 
number of queries, synchronization manager 204 (FIG. 

2) determines if the date and time attached to the queue 
5 falls is valid, i.e. the date and time falls within the specific 

interval allocated for the data distribution. 

If the date and time falls within the specific interval 
allocated for the data distribution, synchronization man- 
ager 204 (FIG. 2) proceeds to step 404 for concurrence 
10 check and subsequent distribution. 

If the data and time does not fall within the specific 
interval allocated for data transmission, synchronization 
manager 204 (FIG. 2) proceeds to step 422 to select 
and group deferred queues. Thereafter synchronization 
manager 204 proceeds to step 404 for concurrence 
check and subsequent distribution. 

FIG. 5 illustrates the distribution process to sub- 
scribing systems 104 (FIG. 1) that was discussed in re- 
lationship to step 410 in more details. 

In step 500, as data distributor 212 (FIG. 2) trans- 
mits the update transaction query, the status of the 
sending query is logged for records with date and time. 
The update query that is sent to subscribing systems 
104 (FIG. 1 ) requires acknowledgment and prompts da- 
tabase engine 116, 11 8 or 1 20 (FIG 1 ) to return the ac- 
knowledgment. The acknowledgment indicates whether 
the subscribing systems have received the transmitted 
query with the following symbols: query update complet- 
ed fUc") or query update failed ("Uf ") ; 

In step 502, response handler 21 4 (FIG. 2) receives 
the acknowledgment from database engine 116, 118 or 
120, and logs the acknowledgment. 

In step 504, response handler 214 (FIG. 2) triggers 
synchronization manager 204 (FIG. 2) to maintain cur- 
rent information with the acknowledgment. 

Synchronization manager 204 (FIG. 2) in step 506 
determines whether the acknowledgment for each que- 
ry indicates query transmission was successful and 
completed. 

If acknowledgment indicates that query transmis- 
sion is a failure, synchronization manager 204 (FIG. 2) 
in step 508 moves the queue so that such retry queue 
will be picked up for transmission again in step 326 (FIG. 

3) . 

If synchronization manager 204 (FIG. 2) receives 
the acknowledgment indicating successful transmis- 
sion, synchronization manager 204 (FIG. 2) in step 510 
removes the queue from update queue 206 (FIG. 2), and 
decreases the concurrence control level in concurrence 
controller 210 (FIG. 2) by one. 

It will be understood that the foregoing is only illus- 
trative of the principles of the invention and that various 
modifications can be made by those skilled in the art 
without departing from the scope and spirit of the inven- 
tion. 
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Claims 

1 . A database network having a primary database and 
a plurality of heterogeneous subscribing databases 
for replicating data updates of said primary data- 
base in said heterogeneous subscribing databases, 
comprising: 

a database engine connected to said primary 
database for capturing said data updates in 
said primary database; 

a query manager connected to said database 
engine for generating queries, each of said 
queries being translated based on a specified 
format for each of said heterogeneous sub- 
scribing databases; and 
a data distributor connected to said query man- 
ager for distributing said translated queries to 
said heterogeneous subscribing databases. 

2. The database network in claim 1 , further compris- 
ing: 

a subscription controller for verifying that at 
least one of said subscribing databases needs 
said updates; and 

said query manager for generating queries only 
for said at least one of said subscribing data- 
bases that needs said updates. 

3. The database network in claim 1 , further comprising 
a synchronization manager connected to said query 
manager for controlling said query distribution. 

4. The database network in claim 3, further comprising 
a plurality of update queues connected to said que- 
ry manager for storing said queries generated from 
said query manager. 

5. The database network in claim 4, further comprising 
a queue controller connected to said update queues 
for prioritizing said queues in distribution. 

6. The database network in claim 5, further compris- 
ing: 

a concurrence controller connected to said 
queue controller for opening distribution chan- 
nels of said distributor; and 
said synchronization manager enables said 
concurrence controller only if said queues can 
be handled by said concurrence controller. 

7. The database network in claim 3, furth r compris- 
ing: 

a response handler for verifying completion of 
said updates in said subscribing databases; 



and 

said.synchronization manager for redistributing 
said queries if said updates in said subscribing 
databases fail. 

5 

8. A method ol replicating data updates of a primary 
database in a plurality of heterogeneous subscrib- 
ing databases in a database network having said 
primary database and said heterogeneous sub- 

io scribing databases, comprising: 

capturing said data updates in said primary da- 
tabase; 

generating queries, each of said queries being 
15 translated based on a specified format for each 

of said heterogeneous subscribing databases; 
and 

distributing said translated queries to said het- 
erogeneous subscribing databases. 

20 

9. The method in claim 8, further comprising the steps 
of: 

verifying that at least one of said subscribing 
25 databases needs said updates; and 

generating queries only for said at least one of 
said subscribing databases that needs said up- 
dates. 

30 10. The method in claim 8, further comprising the step 
of controlling said query distribution. 

11 . The method in claim 10, further comprising the step 
of storing said queries generated from said query 

35 manager. 

12. The method in claim 11 , further comprising the step 
of prioritizing said queues in distribution. 

40 13. The method in claim 12, further comprising the 
steps of: 

opening distribution channels of said distribu- 
tor; and 

4£ enabling said distribution channels only if said 

queues can be handled by said distribution 
channels. 

14. The method in claim 10, further comprising the 
so steps of: 

verifying completion of said updates in said 
subscribing databases; and 
redistributing said qu ri s if said updates in 
55 said subscribing databas s fail. 
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